REMARKS 

This Preliminary Amendment cancels the original claims, without prejudice, in the 
underlying PCX Application No. PCT/DE03/01685. The Preliminary Amendment also adds new 
claims 1 1-20. The new claims conform the claims to U.S. Patent and Trademark Office mles and 
do not add new matter to the application. 

In accordance with 37 C.F.R. § 1.125(b), the Substitute Specification (including 
the Abstract) contains no new matter. The amendments reflected in the Substitute Specification 
(including Abstract) are to conform the Specification and Abstract to United States Patent and 
Trademark Office rules or to correct informalities. As required by 37 C.F.R. §§ 1.121(b)(3)(ii) 
and 1.125(c), a Marked-Up Version of the Substitute Specification comparing the Specification 
of record and the Substitute Specification also accompanies this Preliminary Amendment. 
Approval and entry of the Substitute Specification (including Abstract) are respectfiiUy 
requested. 

The underlying PCT Application No. PCT/DE03/01685 includes an International 
Search Report, dated October 10, 2003, a copy of which is submitted herewith. 

Applicants assert that the subject matter of the present application is new, non- 
obvious, and usefiil. Prompt consideration and allowance of the application are respectfiiUy 
requested. 

RespectfiiUy submitted. 



KENYON & KENYON 



Dated: W^^V By: 1^-^^ %^ 

Richard f^. 1 



Mayer 
(Reg. No. 22.490) 

One Broadway 

New York, NY 10004 

(212) 425-7200 



NYOl 678704 vl 



4 



10/517744 

DT05 Rec'd PCT/PTO 0 9 DEC 20W 

[10191/3625] 

METHOD AND DEVICE FOR DRIVER INFORMATION AND 
FOR A REACTION WHEN LEAVING THE TRAFFIC LANE 



Background Informatio n F/e/J Of The Invention 

The present invention relates to a method and a device for driver information and a reaction 
upon departure from the traffic lane, or, more accurately, in response to the leaving of the 
lane that is to be expected. 

5 

Background In formation 

Warning systems, which warn the driver when he leaves the lane, or rather, threatens to leave 
the lane, are known in the related art. EP 1071^30 Al, European Published Patent 
Application No. 1074430, for example, shows a procedvire in which, based on image sensor 
10 systems, the lane in which the vehicle is moving is ascertained, and the driver is warned when 
the vehicle leaves the lane or threatens to do so. Information on the exact boundary 
conditions, under, which such a warning is given, is not stated. In particular, the respective 
driving situation is not taken into consideration, so that disagreeable false warnings to the 
driver may occur. 

15 

From &EGerman Published Patent Application No. 196 37 053-Arlr it is known that one may 
detect objects on a neighboring traffic lane, with the aid of an angle-resolving distance and 
speed sensor, preferably a radar or laser sensor, and ascertain their travel direction in the light 
of the relative speed of these objects. 

20 

Summarv of the Invention 

By taking into consideration the driving situation for triggering the warning and/or the 
vehicle intervention (reaction) upon leaving the traffic lane, false warnings or false reactions 
are effectively avoided. Warnings of the driver or reactions for avoiding leaving the traffic 
25 lane therefore take place only in response to a relatively certainly recognized danger 
situation. Thus by using the measure described, the utility and the acceptance of a 
corresponding system are greatly enhanced. 

In an advantageous manner, the driving situation on the neighboring traffic lane is recognized 
30 in the light of the kind of traffic lane markings and/or the type of traffic in whose direction 
the leaving of the lane is to be expected. In one concrete application, when a broken line is 
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recognized as a traffic lane marking, no warning is undertaken, or a more inconspicuous 
warning and no vehicle intervention. In another application, a neighboring lane for traffic in 
the same travel direction is regarded as a travel situation in which a warning and a reaction 
upon leaving the traffic lane may be undesired, and consequently suppression of the warning 
5 and the reaction takes place, or an inconspicuous waming is generated. By contrast to this, 
upon recognition of an imbroken line, or, in response to a neighboring lane for traffic in the 
opposite direction, a dangerous situation having a high probability should be present in 
response to leaving, or threatening to leave the traffic lane, so that, in such a situation, a 
waming and/or a reaction is meaningful. If the vehicle crosses a line in the direction of a lane 
10 which is known, e.g. from the information of a navigation system as a breakdown lane, the 
waming may be delayed or undertaken using less intensity. 

Also advantageous is a combination of the criteria of the kind of traffic lane marking and the 
type of traffic situation in the neighboring lane. In that case, given a broken line and a 
15 neighboring traffic lane for traffic in the driving direction, the waming or the reaction is 
suppressed, or an inconspicuous waming is generated, whereas in the case of a broken line 
and a neighboring lane for traffic in the opposite direction, as well as in the case of unbroken 
markings, a waming and a reaction take place. 

20 In an especially advantageous manner, the data that lead to the derivation of the traffic 

situations are ascertained from image sensor signals, and perhaps in supplementary fashion 
from distance sensor signals or from data of a navigation system. The expenditure for 
implementing the procedure described is decreased by using sensors and data that are present 
anyway. 

25 

In an advantageous manner, by using sensor systems that ascertain data forwards, backwards 
and/or sideways with respect to objects present, the accuracy of the waming and the reaction 
may be further enhanced. In this case, a waming and a reaction take place if the traffic lane is 
left or leaving the lane is threatened and, possibly in response to detected broken marking 
30 lines, there are indeed objects on the neighboring lane in the surroundings of the vehicle, 
which are particularly moving in the same travel direction. 

Additional advantages result from the following description of exemplary embodiments and 
from the dependent claims. 

35 
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Brief Description of the Drawings 

Th e pres e nt invention is e xplained in greater detail bolow, on th e basis of tho Gp e cific 
e mbodim e nts depict e d in th e drawings. 

Figure 1 shows, in this context, a block diagram of a device for warning a driver and 
5 for a 

reaction when the vehicle threatens to leave the lane. 

Figure 2 shows a flow chart is sketched in which a preferred procedure for triggering the 
driver warning is shown. 
10 This procoduro is made cl e ar e r in th e light of the sk e tches of 

Figur e 3. 

Figure 3a shows a first illustration of a threat of crossing a solid line, 
15 Figure 3b shows a second illustration of a threat of crossing a solid line. 
Figure 3c shows a first illustration of a threat of crossing a broken line. 
Figure 3d shows a second illustration of a threat of crossing a broken line. 

20 

Detailed Description of th e Ex e mplary Embodim e nts 

Figure 1 shows a device used for warning the driver and for a reaction in response to future 
leaving of the lane of the vehicle. What is shown is a control or evaluation unit 10, which has 
an input circuit 12, a microcomputer 14 and an output circuit 16. These elements are 

25 interconnected via a bus system for mutual data exchange. Input lines from various 

measuring devices, via which measuring signals and measuring data are transmitted, are 
supplied to input circuit 12. A first input line 20 connects input circuit 12 to an image sensor 
system 22, which picks up the scene in fi-ont of the vehicle. Corresponding image data are 
transmitted via input line 20. Furthermore, input lines 24 to 28 are provided which connect 

30 input circuit 12 to measuring devices 30 to 34. These measuring devices are, for example, 
measuring devices for measuring the vehicle speed, for recording the steering angle and for 
recording additional operating variables of the vehicle, which have meaning in connection 
with the procedure described below. At least one warning device 38 is activated via output 
circuit 16 and output line 36, for instance, a warning lamp and/or a loudspeaker for an 

35 acoustical warning or a voice response and/or a display for displaying an image, with the aid 
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of which the driver is informed and/or warned before the departure, that is to be feared, from 
the lane, according to the procedure shown below. In addition, or alternatively, it is provided 
in some exemplary embodiments that one should activate a controlling system 42 via output 
circuit 16 and an output line 40, which automatically leads back into the lane, for instance by 
5 an intervention in the steering of the vehicle, and thus prevents leaving the lane. 

In the preferred exemplary embodiment, the recognition of the lane markings is made by an 
image sensor system, preferably a camera system. By analysis of the pictures taken, 
especially in the light of the gray-scale value distribution, an image evaluation unit 

10 recognizes edge markings whose pattern is ascertained as a mathematical function and a 

differentiation is made as to whether the lines are broken or solid. The image sensor system 
is also used in one embodiment to observe objects in the surroundings of the traffic lane of 
the vehicle, and consequently also in neighboring lanes (left or right). In this context, same 
as above, customary object recognition methods are used. In the light of successive 

15 exposures and the speed of one's own vehicle, based on these data, one may ascertain whether 
a detected object in a neighboring lane is traveling in the same direction or in the opposite 
direction. In this way one may record the travel situation (type of marking and or kind of 
neighboring lane and/or traffic situation). Instead of the image sensor system, in other 
embodiments other sensors are used to record the driving situation, such as radar sensors, 

20 laser sensors, etc. Furthermore, in other embodiments sensors are present which detect 

objects all the way aroimd the vehicle and which ascertain the distances from these objects. 
In these cases, one may not only ascertain the travel direction in a neighboring lane, but 
rather the actual presence of vehicles and their direction of movement in the neighboring 
lane. 

25 

Based on the data concerning at least one travel lane edge marking, whose pattem is shown 
as a mathematical equation, as well as the future trajectory of one's own vehicle ascertained, 
for example, based on the steering angle, etc, in response to intersection or inadmissible close 
approach of these two functions, it is determined that leaving the lane is to be expected. If 
30 this is the case, and if there is a travel situation in which one may assume a heightened 

potential of danger, a waming is emitted to the driver, acoustically and/or optically and/or 
haptically, or by a purposeful intervention in the course of the vehicle, such as by a steering 
intervention, a correction of the course is undertaken. In this context, the waming is clear, 
such as loud tones, fading in a large, possibly colored waming notice, etc. 

35 
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If a solid line is recognized as edge an marking, such as a solid center line or edge line, if 
there is a threat of crossing these edge lines, a warning and a reaction are undertaken. In the 
case of broken lines, such as broken center lines, in one preferred exemplary embodiment no 
warning and no reaction are generated. In another embodiment, in this case, an 
5 inconspicuously fashioned warning takes place, such as a soft tones, a smaller optical 
representation, less conspicuous colors, etc. 

If observation of the neighboring lane yields no information about the direction of travel in 
the neighboring lane, no (less conspicuous) warning and no reaction are generated. If the 
10 travel direction of traffic in the neighboring lane is known, no waming is emitted or the 

waming is designed to be inconspicuous if the driver changes to a traffic lane having traffic 
in the same direction. If he changes to a lane having traffic in the opposite direction, a 
waming is emitted. The vehicle intervention (reaction) takes place in one case and not in the 
other. 

15 

In another exemplary embodiment it is ascertained fi-om the data of a navigation system that 
are present whether the vehicle is on a road on which the respective neighboring lane is being 
used by vehicles going in the same direction, or which represent traffic in the opposite 
direction. 

20 

In still another embodiment, if there is a solid line, a waming is always given, and in the case 
of a broken line only if the neighboring lane was recognized as being a lane for oncoming 
traffic. 

25 Depending on the exemplary embodiment and the outfitting with sensors, it may be provided 
that, in the case of a broken line a waming is emitted if, via the sensor system, actually 
present objects in the neighboring lane are detected in the same direction of travel, near the 
vehicle. 

30 In one exemplary embodiment, a driving situation that has once been recognized is assimied 
imtil other information is present. If, for example, it is recognized that the vehicle is on its 
way in a traffic lane having oncoming traffic, a traffic lane having traffic in the same 
direction is assumed only if this is reliably recognized. The corresponding thing is true for 
recognizing a driving situation having oncoming traffic. 

35 
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If a warning is suppressed, and if it is recognized at the same time that the vehicle will 
probably leave the lane, then, in one preferred exemplary embodiment, in this situation, a 
blinker noise is automatically put out for the driver as inconspicuous information, 
In one preferred exemplary embodiment, in addition in this situation, the respective blinker is 
5 switched on in order to inform other traffic participants. 

The above described procedure is implemented using programs which are implemented on 
microcomputer 14. The latter generates the output signals for the warning or vehicle 
intervention, with the aid of the programs and according to the input signals supplied. In the 
10 light of a flow diagram, a preferred exemplary embodiment of the procedure represented 
above is shown below. 

The program shown in Figure 2 is run through when it has been recognized that there is a 
danger that the vehicle will leave its lane. In first step 100 it is determined whether the 

15 marking, whose crossing is being threatened, is a solid or a broken line. This is done based on 
the corresponding information firom the image evaluation imit. If the edge marking is a solid 
line, then, according to step 108, a driver warning/vehicle intervention takes place, for 
example, by acoustical and/or optical and/or haptic warning, or by intervention in the vehicle 
steering. However, if, in step 100, it was recognized that the lane marking, the crossing of 

20 which is being threatened, is a broken line, then in step 102 the traffic situation with respect 
to the neighboring lane, to which one could change, is checked. As was described above, it is 
checked, using a relative speed signal, whether the traffic moving on the neighboring lane is 
on its way in the same travel direction or in the opposite direction. In this context, a marker is 
set if, possibly several times in a row, objects in the respective neighboring lane have been 

25 recognized that are oncoming. If the marker has been set, step 108 follows with a driver 

warning/vehicle intervention. The marker is canceled when it has been certainly recognized 
(i.e. several times in a row) that objects in the neighboring lane are traveling in the same 
direction. If the marker has been canceled, in step 104, no warning/vehicle intervention is 
undertaken; in one embodiment, automatically a blinker noise generated or an optical 

30 indication given. Optionally then, in step 106 the corresponding blinker is switched on in 
order to inform other traffic participants. After step 106 or 108, the program is continued 
again for the ascertainment of the danger of crossing an edge marking. 
In other embodiments, in response to a recognized broken line and traffic in the driving 
direction, a warning is generated if an object has been recognized close to the vehicle, and/or 

35 in the case of oncoming traffic, if an oncoming vehicle is actually recognized. In the latter 
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case, an inconspicuous, or no warning is emitted if the neighboring traffic lane is actually for 
oncoming traffic, but no actual oncoming traffic has been recognized. 



In Figure 3 various driving situations are shown in which, upon leaving the lane or when 
5 there is a threat of leaving the lane, a driver waming takes place or does not. In Figures 3a 
and 3b there is a threat of crossing a solid line, so that, in this case, independently of the 
driving situation, a waming occurs. In Figures 3c and 3d there is a threat of crossing a 
broken line. In this case, as shown above, a waming occurs if there is oncoming traffic in the 
neighboring lane or if oncoming traffic is to be expected. This situation is depicted in Figure 
10 3d, in which, therefore, a driver waming takes place. In Figure 3c, on the other hand, the 
vehicle in the neighboring lane is traveling in the same direction, so that, in this case, the 
waming may be omitted or is made less conspicuous (more softly, blinker noise, smaller, 
etc). 

15 In one additional specific embodiment, the waming is delayed or vmdertaken using less 

intensity if the vehicle crosses a line in the direction of a lane which is known, e.g. firom the 
information of a navigation system as a breakdown lane. 
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Abstract Of The Disclosure 
A method and a device are proposed for driver information and for a reaction upon the 
leaving of the traffic lane by the vehicle. The diver warning and the driver intervention are a 
function, in this context, of the driving situation, especially of the type of an edge marking of 
the traffic lane. 
(Figur e 2) 
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